Skip to content

ci: Add engine compatibility check for PRs#1356

Draft
d3xter666 wants to merge 3 commits intomainfrom
engine-compatibility-checks
Draft

ci: Add engine compatibility check for PRs#1356
d3xter666 wants to merge 3 commits intomainfrom
engine-compatibility-checks

Conversation

@d3xter666
Copy link
Copy Markdown
Member

Reusable workflow that runs npm ci --dry-run against boundary Node versions and posts a consolidated PR comment with incompatible packages. Adds blocked label on failure with detailed Github Bot comment with the failing packages and their node engines.

Approaches evaluated:

  • npm ci --engine-strict: runtime check only, stops at first failure, misses remaining incompatible packages
  • check-engine (npm): only checks running Node against root engines, doesn't inspect dependency tree
  • check-engine-light (npm): range-vs-range via semver.subset(), but v0.x, only 4 releases, unstable
  • ls-engines (npm): range-vs-range capable, but v0.x, 17-month release gap, 7 prod dependencies

Selected approach: single-job nvm loop using npm ci --dry-run (without --engine-strict). Reports all EBADENGINE warnings as a table in one PR comment. No extra dependencies, no matrix, no artifacts — just shell and tools already on the runner.
Note: NVM seems to come bundled in the GitHub action images

PoC of the implementation: d3xter666/test-ci#1
JIRA: CPOUI5FOUNDATION-1225

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

✅ Engine Compatibility

Node Result Incompatible Packages
v22.20.0 ✅ Pass
v22.22.2 ✅ Pass
v24.0.0 ✅ Pass
v24.14.1 ✅ Pass

@d3xter666
Copy link
Copy Markdown
Member Author

d3xter666 commented Apr 9, 2026

Note: This flow can be ruesed accross repositories the following way:

uses: UI5/cli/.github/workflows/engine-compat.yml@main
with:
  node-versions: '20, 22, 24'

The caller uses owner/repo/.github/workflows/file.yml@ref syntax instead of ./.github/workflows/...
No copying needed. The @ main can also be a tag (@ v1) or SHA for pinning.

I gave it a try in an external repositorty and it seems to be working correctly:

@d3xter666 d3xter666 requested a review from a team April 9, 2026 11:54
@RandomByte
Copy link
Copy Markdown
Member

This is a good idea, but I'm wondering how we can ignore dev-dependencies as discussed last week?

The check-engine-light implementation seems rather simple. If the lack of usage is really a concern for us we can also consider implementing such a tool ourselves. I had already started doing that, but then found that my name of choice "check-engine-light", was already taken by a tool with the very same functionality. In general, the check is really easy to implement and more efficient than the proposal here since we only need to parse the lockfile and not install any packages.

@d3xter666
Copy link
Copy Markdown
Member Author

d3xter666 commented Apr 14, 2026

This is a good idea, but I'm wondering how we can ignore dev-dependencies as discussed last week?

The check-engine-light implementation seems rather simple. If the lack of usage is really a concern for us we can also consider implementing such a tool ourselves. I had already started doing that, but then found that my name of choice "check-engine-light", was already taken by a tool with the very same functionality. In general, the check is really easy to implement and more efficient than the proposal here since we only need to parse the lockfile and not install any packages.

If it's for the control of dependencies, we can leverage the --omit or --include flag.
Something that we also need to consider is how to redistribute this check accross all the repositories we own.
With this approach we easily update/modify the script in a single place and it automatically appears on the other repoisories. It can be "forzen" for certain repositories by providing a specific tag/hash/branch. Also this solution uses native and built-in tools that will always work. Direct access to PRs via gh CLI.

The issue with the listed packages are actually what we currently have- potential lack of update in case of security issues, not a major version (the major version release might not fit our engine).

From that perspective, if we want to use a custom script, we have the following options:

  • Parse package-lock.json / npm-shrinkwrap.json (how to identify not prod dependencies?)
  • Use shell with npm ls --json [--omit <dev|optional|peer>] command, so we can leverage Node's parsing directly without manually parsing anything or including dependencies like arborist, for example

Then we'll simply use semver in order to compare packages' versions against our package.json
Then, we have the following chalenges to address:

  • What we want to do with this information?
    • Post to GitHub (maybe directly invoke gh in shell)
    • Exit code?
    • Something else
  • How to redistribute the script?

@d3xter666 d3xter666 changed the title feat: Add engine compatibility check for PRs ci: Add engine compatibility check for PRs Apr 14, 2026
@d3xter666 d3xter666 marked this pull request as draft April 14, 2026 11:45
@d3xter666
Copy link
Copy Markdown
Member Author

This is a good idea, but I'm wondering how we can ignore dev-dependencies as discussed last week?
The check-engine-light implementation seems rather simple. If the lack of usage is really a concern for us we can also consider implementing such a tool ourselves. I had already started doing that, but then found that my name of choice "check-engine-light", was already taken by a tool with the very same functionality. In general, the check is really easy to implement and more efficient than the proposal here since we only need to parse the lockfile and not install any packages.

If it's for the control of dependencies, we can leverage the --omit or --include flag. Something that we also need to consider is how to redistribute this check accross all the repositories we own. With this approach we easily update/modify the script in a single place and it automatically appears on the other repoisories. It can be "forzen" for certain repositories by providing a specific tag/hash/branch. Also this solution uses native and built-in tools that will always work. Direct access to PRs via gh CLI.

The issue with the listed packages are actually what we currently have- potential lack of update in case of security issues, not a major version (the major version release might not fit our engine).

From that perspective, if we want to use a custom script, we have the following options:

  • Parse package-lock.json / npm-shrinkwrap.json (how to identify not prod dependencies?)
  • Use shell with npm ls --json [--omit <dev|optional|peer>] command, so we can leverage Node's parsing directly without manually parsing anything or including dependencies like arborist, for example

Then we'll simply use semver in order to compare packages' versions against our package.json Then, we have the following chalenges to address:

  • What we want to do with this information?

    • Post to GitHub (maybe directly invoke gh in shell)
    • Exit code?
    • Something else
  • How to redistribute the script?

Just created a separate version with check-engine-light: #1360
Of course, we can quite simplify it and just reuse the check-engine-light package. I remember though that we discussed we just want the information whether certain package is ok or not. With the current setup we have the information available, but it's not blocking the PR from being merged.
If we use the pure check-engine-light package if everyhting is ok, then it will complete silently, but if there's an incompatibility it will exit with 1 and this will fail the general ci checks. This will have (almost) the same effect and hit the same boundaries as it is currently with npm ci --engine-strict

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants